REFRACTIVE: An Open Source Tool to Extract Knowledge from Syntactic and Semantic Relations

The extraction of semantic propositions has proven instrumental in applications like IBM Watson (Ferrucci, 2012) and in Google’s knowledge graph (Singhal, 2012). One of the core components of IBM Watson is the PRISMATIC knowledge base consisting of one billion propositions extracted from the English version of Wikipedia and the New York Times (Fan et al., 2010). However, extracting the propositions from the English version of Wikipedia is a time-consuming process. In practice, this task requires multiple machines and a computation distribution involving a good deal of system technicalities. In this paper, we describe REFRACTIVE, an open-source tool to extract propositions from a parsed corpus based on the Hadoop variant of MapReduce. While the complete process consists of a parsing part and an extraction part, we focus here on the extraction from the parsed corpus and we hope this tool will help computational linguists speed up the development of applications.